2021-03-11T19:03:05.425 INFO:root:teuthology version: 1.1.0-b7dbd64f 2021-03-11T19:03:05.427 INFO:teuthology.results:subset: '111/120000' 2021-03-11T19:03:05.427 INFO:teuthology.results:seed: '7595' 2021-03-11T21:42:12.447 INFO:root:teuthology version: 1.1.0-b7dbd64f 2021-03-11T21:42:12.477 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-12T08:35:11.297 INFO:teuthology.results:Tests finished! gathering results... 2021-03-12T08:35:50.695 INFO:scrape:Found 320 jobs 2021-03-12T08:35:55.866 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:55.869 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:55.871 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:55.871 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:55.992 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:55.994 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:55.996 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:55.997 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.123 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.125 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.127 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.128 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.257 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.259 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.261 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.262 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.379 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.381 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.383 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.383 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.494 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.496 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.498 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.499 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.611 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.613 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.615 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.615 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.730 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.732 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.734 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.735 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:56.848 WARNING:scrape:Ignoring malparsed backtrace: ceph version 15.2.9-126-g5e61b158 (5e61b15880d42cb4248c17ea9494bc11be2b7a78) octopus (stable) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7efe2738c7da] , 2: (()+0x27a9f4) [0x7efe2738c9f4] 2021-03-12T08:35:56.850 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:23:40.557 INFO:teuthology.orchestra.run.smithi095.stdout: , dumped all , 2021-03-11T21:23:40.569 INFO:teuthology.orchestra.run.smithi095.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2021-03-11T21:23:39.981466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30157973406,"num_objects":10152,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":41,"num_read":22949,"num_read_kb":1614059,"num_write":53063,"num_write_kb":23883729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49218,"ondisk_log_size":49218,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562196280,"kb_used":43627968,"kb_used_data":518568856,"kb_used_omap":73516,"kb_used_meta":0,"kb_avail":518568312,"statfs":{"total":575688990720,"available":531013951488,"internally_reserved":557056,"allocated":531014508544,"data_stored":531014508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75280515,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16553156,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":12,"num_read":568,"num_read_kb":601977,"num_write":287,"num_write_kb":168105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":591,"num_flush":33,"num_flush_kb":69445,"num_evict":28,"num_evict_kb":66110,"num_promote":56,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001707"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283597+0000","last_change":"2021-03-11T21:20:25.600556+0000","last_active":"2021-03-11T21:23:10.283597+0000","last_peered":"2021-03-11T21:23:10.283597+0000","last_clean":"2021-03-11T21:23:10.283597+0000","last_became_active":"2021-03-11T21:20:25.600245+0000","last_became_peered":"2021-03-11T21:20:25.600245+0000","last_unstale":"2021-03-11T21:23:10.283597+0000","last_undegraded":"2021-03-11T21:23:10.283597+0000","last_fullsized":"2021-03-11T21:23:10.283597+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283389+0000","last_change":"2021-03-11T21:20:25.216605+0000","last_active":"2021-03-11T21:23:10.283389+0000","last_peered":"2021-03-11T21:23:10.283389+0000","last_clean":"2021-03-11T21:23:10.283389+0000","last_became_active":"2021-03-11T21:20:25.216253+0000","last_became_peered":"2021-03-11T21:20:25.216253+0000","last_unstale":"2021-03-11T21:23:10.283389+0000","last_undegraded":"2021-03-11T21:23:10.283389+0000","last_fullsized":"2021-03-11T21:23:10.283389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'5843","reported_seq":"8123","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:32.282034+0000","last_change":"2021-03-11T21:20:35.238244+0000","last_active":"2021-03-11T21:23:32.282034+0000","last_peered":"2021-03-11T21:23:32.282034+0000","last_clean":"2021-03-11T21:23:32.282034+0000","last_became_active":"2021-03-11T21:20:35.237916+0000","last_became_peered":"2021-03-11T21:20:35.237916+0000","last_unstale":"2021-03-11T21:23:32.282034+0000","last_undegraded":"2021-03-11T21:23:32.282034+0000","last_fullsized":"2021-03-11T21:23:32.282034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5843,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":86958697,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":2016,"num_read_kb":44469,"num_write":5145,"num_write_kb":1191332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":4104,"num_flush":942,"num_flush_kb":2728665,"num_evict":848,"num_evict_kb":2449475,"num_promote":1042,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"33'6532","reported_seq":"8587","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.855945+0000","last_change":"2021-03-11T21:20:33.662718+0000","last_active":"2021-03-11T21:23:33.855945+0000","last_peered":"2021-03-11T21:23:33.855945+0000","last_clean":"2021-03-11T21:23:33.855945+0000","last_became_active":"2021-03-11T21:20:33.662420+0000","last_became_peered":"2021-03-11T21:20:33.662420+0000","last_unstale":"2021-03-11T21:23:33.855945+0000","last_undegraded":"2021-03-11T21:23:33.855945+0000","last_fullsized":"2021-03-11T21:23:33.855945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6532,"ondisk_log_size":6532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7203184832,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":3748,"num_read_kb":480661,"num_write":8129,"num_write_kb":4611133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283548+0000","last_change":"2021-03-11T21:20:25.215786+0000","last_active":"2021-03-11T21:23:10.283548+0000","last_peered":"2021-03-11T21:23:10.283548+0000","last_clean":"2021-03-11T21:23:10.283548+0000","last_became_active":"2021-03-11T21:20:25.215311+0000","last_became_peered":"2021-03-11T21:20:25.215311+0000","last_unstale":"2021-03-11T21:23:10.283548+0000","last_undegraded":"2021-03-11T21:23:10.283548+0000","last_fullsized":"2021-03-11T21:23:10.283548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'5281","reported_seq":"7326","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.572872+0000","last_change":"2021-03-11T21:20:35.357290+0000","last_active":"2021-03-11T21:23:33.572872+0000","last_peered":"2021-03-11T21:23:33.572872+0000","last_clean":"2021-03-11T21:23:33.572872+0000","last_became_active":"2021-03-11T21:20:35.356901+0000","last_became_peered":"2021-03-11T21:20:35.356901+0000","last_unstale":"2021-03-11T21:23:33.572872+0000","last_undegraded":"2021-03-11T21:23:33.572872+0000","last_fullsized":"2021-03-11T21:23:33.572872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5281,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":118746843,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":1855,"num_read_kb":7300,"num_write":4753,"num_write_kb":1132912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":2973,"num_flush":843,"num_flush_kb":2455561,"num_evict":726,"num_evict_kb":2122524,"num_promote":912,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'6814","reported_seq":"9062","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.167696+0000","last_change":"2021-03-11T21:20:33.230675+0000","last_active":"2021-03-11T21:23:34.167696+0000","last_peered":"2021-03-11T21:23:34.167696+0000","last_clean":"2021-03-11T21:23:34.167696+0000","last_became_active":"2021-03-11T21:20:33.230477+0000","last_became_peered":"2021-03-11T21:20:33.230477+0000","last_unstale":"2021-03-11T21:23:34.167696+0000","last_undegraded":"2021-03-11T21:23:34.167696+0000","last_fullsized":"2021-03-11T21:23:34.167696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6814,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7563788702,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":3856,"num_read_kb":357794,"num_write":8431,"num_write_kb":4972211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"26","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283412+0000","last_change":"2021-03-11T21:20:25.216477+0000","last_active":"2021-03-11T21:23:10.283412+0000","last_peered":"2021-03-11T21:23:10.283412+0000","last_clean":"2021-03-11T21:23:10.283412+0000","last_became_active":"2021-03-11T21:20:25.216142+0000","last_became_peered":"2021-03-11T21:20:25.216142+0000","last_unstale":"2021-03-11T21:23:10.283412+0000","last_undegraded":"2021-03-11T21:23:10.283412+0000","last_fullsized":"2021-03-11T21:23:10.283412+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'5109","reported_seq":"7164","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:33.429424+0000","last_change":"2021-03-11T21:20:35.357442+0000","last_active":"2021-03-11T21:23:33.429424+0000","last_peered":"2021-03-11T21:23:33.429424+0000","last_clean":"2021-03-11T21:23:33.429424+0000","last_became_active":"2021-03-11T21:20:35.357084+0000","last_became_peered":"2021-03-11T21:20:35.357084+0000","last_unstale":"2021-03-11T21:23:33.429424+0000","last_undegraded":"2021-03-11T21:23:33.429424+0000","last_fullsized":"2021-03-11T21:23:33.429424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:34.229189+0000","last_clean_scrub_stamp":"2021-03-11T21:20:34.229189+0000","log_size":5109,"ondisk_log_size":5109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":123477019,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":1790,"num_read_kb":854,"num_write":4546,"num_write_kb":1106372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":2584,"num_flush":809,"num_flush_kb":2373831,"num_evict":685,"num_evict_kb":2019240,"num_promote":894,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"33'6779","reported_seq":"8962","reported_epoch":"33","state":"active+clean+scrubbing+deep","last_fresh":"2021-03-11T21:23:39.657612+0000","last_change":"2021-03-11T21:23:38.865411+0000","last_active":"2021-03-11T21:23:39.657612+0000","last_peered":"2021-03-11T21:23:39.657612+0000","last_clean":"2021-03-11T21:23:39.657612+0000","last_became_active":"2021-03-11T21:20:33.662146+0000","last_became_peered":"2021-03-11T21:20:33.662146+0000","last_unstale":"2021-03-11T21:23:39.657612+0000","last_undegraded":"2021-03-11T21:23:39.657612+0000","last_fullsized":"2021-03-11T21:23:39.657612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6779,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7493885224,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":3771,"num_read_kb":363039,"num_write":8207,"num_write_kb":4937614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283257+0000","last_change":"2021-03-11T21:20:25.215639+0000","last_active":"2021-03-11T21:23:10.283257+0000","last_peered":"2021-03-11T21:23:10.283257+0000","last_clean":"2021-03-11T21:23:10.283257+0000","last_became_active":"2021-03-11T21:20:25.215253+0000","last_became_peered":"2021-03-11T21:20:25.215253+0000","last_unstale":"2021-03-11T21:23:10.283257+0000","last_undegraded":"2021-03-11T21:23:10.283257+0000","last_fullsized":"2021-03-11T21:23:10.283257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"33'5898","reported_seq":"8197","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:38.414749+0000","last_change":"2021-03-11T21:23:38.414803+0000","last_active":"2021-03-11T21:23:38.414749+0000","last_peered":"2021-03-11T21:23:38.414749+0000","last_clean":"2021-03-11T21:23:38.414749+0000","last_became_active":"2021-03-11T21:20:35.237622+0000","last_became_peered":"2021-03-11T21:20:35.237622+0000","last_unstale":"2021-03-11T21:23:38.414749+0000","last_undegraded":"2021-03-11T21:23:38.414749+0000","last_fullsized":"2021-03-11T21:23:38.414749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5898","last_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_deep_scrub":"33'5898","last_deep_scrub_stamp":"2021-03-11T21:23:38.414748+0000","last_clean_scrub_stamp":"2021-03-11T21:23:38.414748+0000","log_size":5898,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73035101,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":2006,"num_read_kb":996,"num_write":5178,"num_write_kb":1204943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":8,"num_bytes_hit_set_archive":3964,"num_flush":947,"num_flush_kb":2750805,"num_evict":856,"num_evict_kb":2491151,"num_promote":1039,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"33'6962","reported_seq":"9138","reported_epoch":"33","state":"active+clean","last_fresh":"2021-03-11T21:23:34.163648+0000","last_change":"2021-03-11T21:20:33.662662+0000","last_active":"2021-03-11T21:23:34.163648+0000","last_peered":"2021-03-11T21:23:34.163648+0000","last_clean":"2021-03-11T21:23:34.163648+0000","last_became_active":"2021-03-11T21:20:33.662432+0000","last_became_peered":"2021-03-11T21:20:33.662432+0000","last_unstale":"2021-03-11T21:23:34.163648+0000","last_undegraded":"2021-03-11T21:23:34.163648+0000","last_fullsized":"2021-03-11T21:23:34.163648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:32.222612+0000","last_clean_scrub_stamp":"2021-03-11T21:20:32.222612+0000","log_size":6962,"ondisk_log_size":6962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7494896988,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":3907,"num_read_kb":358946,"num_write":8674,"num_write_kb":4727212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215903+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.215464+0000","last_became_peered":"2021-03-11T21:20:25.215464+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283483+0000","last_change":"2021-03-11T21:20:25.215544+0000","last_active":"2021-03-11T21:23:10.283483+0000","last_peered":"2021-03-11T21:23:10.283483+0000","last_clean":"2021-03-11T21:23:10.283483+0000","last_became_active":"2021-03-11T21:20:25.214985+0000","last_became_peered":"2021-03-11T21:20:25.214985+0000","last_unstale":"2021-03-11T21:23:10.283483+0000","last_undegraded":"2021-03-11T21:23:10.283483+0000","last_fullsized":"2021-03-11T21:23:10.283483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"32","state":"active+clean","last_fresh":"2021-03-11T21:23:10.283384+0000","last_change":"2021-03-11T21:20:25.600783+0000","last_active":"2021-03-11T21:23:10.283384+0000","last_peered":"2021-03-11T21:23:10.283384+0000","last_clean":"2021-03-11T21:23:10.283384+0000","last_became_active":"2021-03-11T21:20:25.600309+0000","last_became_peered":"2021-03-11T21:20:25.600309+0000","last_unstale":"2021-03-11T21:23:10.283384+0000","last_undegraded":"2021-03-11T21:23:10.283384+0000","last_fullsized":"2021-03-11T21:23:10.283384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T21:20:24.208239+0000","last_clean_scrub_stamp":"2021-03-11T21:20:24.208239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":402217660,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":41,"num_read":7667,"num_read_kb":53619,"num_write":19622,"num_write_kb":4635559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":27,"num_bytes_hit_set_archive":13625,"num_flush":3541,"num_flush_kb":10308862,"num_evict":3115,"num_evict_kb":9082390,"num_promote":3887,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":4,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22131,"ondisk_log_size":22131,"up":8,"acting":8,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":4,"stat_sum":{"num_bytes":29755755746,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":15282,"num_read_kb":1560440,"num_write":33441,"num_write_kb":19248170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27087,"ondisk_log_size":27087,"up":8,"acting":8,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10390004,"kb_used_data":83309376,"kb_used_omap":12268,"kb_used_meta":0,"kb_avail":83309376,"statfs":{"total":95948165120,"available":85308801024,"internally_reserved":0,"allocated":85308801024,"data_stored":85308801024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12562808,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.38300000000000001,"15min":0.40699999999999997},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60799999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.504},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63900000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.21199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.41799999999999998},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.627,"5min":1.101,"15min":1.101},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.28399999999999997,"15min":0.28799999999999998},"min":{"1min":0.22600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.39100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Mar 11 21:23:20 2021","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27700000000000002,"15min":0.28100000000000003},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.312,"15min":0.32000000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.39100000000000001,"5min":0.502,"15min":0.502},"last":0.54800000000000004}]}]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":948396,"kb_used_data":92751056,"kb_used_omap":12263,"kb_used_meta":0,"kb_avail":92750984,"statfs":{"total":95948165120,"available":94977007616,"internally_reserved":73728,"allocated":94977081344,"data_stored":94977081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12557707,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.5},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.439}]},{"osd":1,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.254,"5min":0.251,"15min":0.251},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.30399999999999999,"15min":0.312},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.254},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.192,"15min":0.192},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.27300000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:26 2021","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.30599999999999999,"15min":0.316},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.224}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":5483768,"kb_used_data":88215612,"kb_used_omap":3712,"kb_used_meta":0,"kb_avail":88215612,"statfs":{"total":95948165120,"available":90332786688,"internally_reserved":0,"allocated":90332786688,"data_stored":90332786688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3801252,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.33100000000000002,"15min":0.35099999999999998},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35899999999999999,"15min":0.372},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.51900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.311,"5min":0.33500000000000002,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46400000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.27000000000000002,"5min":0.29099999999999998,"15min":0.29999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.371,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.373},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.33700000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.36899999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30499999999999999,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.311,"15min":0.32300000000000001},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.38500000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640298,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":10690412,"kb_used_data":83008968,"kb_used_omap":12134,"kb_used_meta":0,"kb_avail":83008968,"statfs":{"total":95948165120,"available":85001183232,"internally_reserved":0,"allocated":85001183232,"data_stored":85001183232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12425780,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.189,"5min":0.22800000000000001,"15min":0.24199999999999999},"min":{"1min":0.129,"5min":0.129,"15min":0.129},"max":{"1min":0.22700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.19800000000000001,"5min":0.23999999999999999,"15min":0.25600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.26800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20499999999999999}]},{"osd":1,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.17899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.30299999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.18099999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34499999999999997,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:22 2021","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":944484,"kb_used_data":92754972,"kb_used_omap":12273,"kb_used_meta":0,"kb_avail":92754896,"statfs":{"total":95948165120,"available":94981013504,"internally_reserved":77824,"allocated":94981091328,"data_stored":94981091328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12567739,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.157,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.26300000000000001,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32300000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.312},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.379,"15min":0.39400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.88,"15min":0.88},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57699999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38800000000000001,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001}]},{"osd":5,"last update":"Thu Mar 11 21:23:21 2021","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.435},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54800000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34399999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":15170904,"kb_used_data":78528872,"kb_used_omap":20864,"kb_used_meta":0,"kb_avail":78528476,"statfs":{"total":95948165120,"available":80413159424,"internally_reserved":405504,"allocated":80413564928,"data_stored":80413564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21365229,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.153,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.27300000000000002,"15min":0.25600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.16400000000000001,"5min":0.157,"15min":0.157},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.28199999999999997,"15min":0.29799999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.437,"5min":1.036,"15min":1.036},"last":0.182}]},{"osd":3,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35099999999999998,"15min":0.36199999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45900000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53800000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.622,"15min":0.622},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Mar 11 21:23:24 2021","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":1.171,"15min":1.171},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.51400000000000001}]}]}]}} 2021-03-12T08:35:56.852 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:24:42.118 INFO:tasks.ceph.mon.a:Stopped , 2021-03-11T21:24:42.120 INFO:tasks.ceph:Checking cluster log for badness... , 2021-03-11T21:24:42.120 DEBUG:teuthology.orchestra.run.smithi095:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2021-03-12T08:35:56.853 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-11T21:25:01.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' , 2021-03-11T21:25:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-03-11T21:25:01.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-03-12T08:35:57.659 INFO:scrape:Found 10 distinct failure reasons 2021-03-12T08:35:57.661 INFO:scrape:Failure: "2021-03-11T21:30:03.440583+0000 mon.a (mon.0) 12 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED)" in cluster log 2021-03-12T08:35:57.661 INFO:scrape:1 jobs: ['5956546'] 2021-03-12T08:35:57.662 INFO:scrape:suites: ['1-install/mimic', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'rados/upgrade/mimic-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}'] 2021-03-12T08:35:57.662 INFO:scrape: 2021-03-12T08:35:57.663 INFO:scrape:Failure: {'Failure object was': {'smithi045.front.sepia.ceph.com': {'changed': False, 'results': ['yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed', 'yum-plugin-fastestmirror-1.1.31-52.el7.noarch providing yum-plugin-fastestmirror is already installed', 'redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed', 'sysstat-10.1.5-18.el7.x86_64 providing sysstat is already installed', 'gdb-7.6.1-115.el7.x86_64 providing gdb is already installed', 'git-all-1.8.3.1-20.el7.noarch providing git-all is already installed', 'python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed', 'gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed', 'libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed', 'openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed', 'boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed', 'xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed', 'xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed', 'gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed', 'parted-3.1-31.el7.x86_64 providing parted is already installed', 'libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed', 'fuse-2.9.2-11.el7.x86_64 providing fuse is already installed', 'fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed', '7:lvm2-2.02.185-2.el7.x86_64 providing lvm2 is already installed', '1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed', 'libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed', 'btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed', 'attr-2.4.46-13.el7.x86_64 providing attr is already installed', '1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed', 'python-nose-1.3.7-1.el7.noarch providing python-nose is already installed', 'mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed', 'ant-1.9.4-2.el7.noarch providing ant is already installed', 'lsof-4.87-6.el7.x86_64 providing lsof is already installed', 'iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed', 'libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed', 'automake-1.13.4-3.el7.noarch providing automake is already installed', 'gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed', 'libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed', 'libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed', 'bc-1.06.95-13.el7.x86_64 providing bc is already installed', 'xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed', 'blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed', '1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed', 'python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed', "No package matching 'qemu-kvm' found available, installed or updated"], 'rc': 126, 'invocation': {'module_args': {'lock_timeout': 30, 'update_cache': False, 'disable_excludes': 'None', 'exclude': [], 'allow_downgrade': False, 'disable_gpg_check': False, 'conf_file': 'None', 'use_backend': 'auto', 'state': 'present', 'disablerepo': [], 'releasever': 'None', 'skip_broken': False, 'autoremove': False, 'download_dir': 'None', 'enable_plugin': [], 'installroot': '/', 'install_weak_deps': True, 'name': ['@core', '@base', 'yum-plugin-priorities', 'yum-plugin-fastestmirror', 'redhat-lsb', 'sysstat', 'gdb', 'git-all', 'python-configobj', 'gcc-c++', 'libedit', 'openssl098e', 'boost-thread', 'xfsprogs', 'xfsprogs-devel', 'gdisk', 'parted', 'libgcrypt', 'fuse', 'fuse-libs', 'lvm2', 'openssl', 'libuuid', 'btrfs-progs', 'attr', 'valgrind', 'python-nose', 'mpich', 'ant', 'lsof', 'iozone', 'libtool', 'automake', 'gettext', 'libuuid-devel', 'libacl-devel', 'bc', 'xfsdump', 'blktrace', 'numpy', 'python-matplotlib', 'qemu-kvm', 'usbredir', 'genisoimage', 'httpd', 'httpd-devel', 'httpd-tools', 'mod_ssl', 'mod_fastcgi-2.4.7-1.ceph.el7', 'libev-devel', 'perl-XML-Twig', 'java-1.6.0-openjdk-devel', 'junit4', 'nfs-utils', 'ncurses-devel', 'python-devel', 'python-virtualenv', 'perl-CPAN', 'podman', 'podman-docker', 'python3'], 'download_only': False, 'bugfix': False, 'list': 'None', 'install_repoquery': True, 'update_only': False, 'disable_plugin': [], 'enablerepo': [], 'security': False, 'validate_certs': True}}, 'msg': "No package matching 'qemu-kvm' found available, installed or updated", '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'changed')"} 2021-03-12T08:35:57.664 INFO:scrape:4 jobs: ['5956765', '5956609', '5956769', '5956605'] 2021-03-12T08:35:57.664 INFO:scrape:suites intersection: ['start}'] 2021-03-12T08:35:57.665 INFO:scrape:suites union: ['fixed-2', 'rados/cephadm/smoke-roleless/{distro/rhel_7', 'rados/cephadm/smoke/{distro/rhel_7', 'start}'] 2021-03-12T08:35:57.665 INFO:scrape: 2021-03-12T08:35:57.666 INFO:scrape:Failure: Command failed on smithi039 with status 5: 'sudo systemctl stop ceph-854797fe-82aa-11eb-9085-001a4aab830c@mon.a' 2021-03-12T08:35:57.666 INFO:scrape:8 jobs: ['5956677', '5956516', '5956523', '5956750', '5956684', '5956622', '5956618', '5956781'] 2021-03-12T08:35:57.667 INFO:scrape:suites intersection: [] 2021-03-12T08:35:57.667 INFO:scrape:suites union: ['fixed-2', 'mode/root', 'msgr/async', 'msgr/async-v1only', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'rados/cephadm/with-work/{distro/ubuntu_18.04_podman', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'start', 'start}', 'task/test_orch_cli}', 'tasks/rados_api_tests}', 'tasks/rados_python}'] 2021-03-12T08:35:57.667 INFO:scrape: 2021-03-12T08:35:57.668 INFO:scrape:Failure: Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress) 2021-03-12T08:35:57.668 INFO:scrape:2 jobs: ['5956564', '5956778'] 2021-03-12T08:35:57.669 INFO:scrape:suites intersection: ['debug/mgr', 'rados/mgr/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_latest}', 'tasks/progress}'] 2021-03-12T08:35:57.669 INFO:scrape:suites union: ['debug/mgr', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'rados/mgr/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_latest}', 'tasks/progress}'] 2021-03-12T08:35:57.670 INFO:scrape: 2021-03-12T08:35:57.670 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi075 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2021-03-12T08:35:57.671 INFO:scrape:2 jobs: ['5956746', '5956591'] 2021-03-12T08:35:57.671 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2021-03-12T08:35:57.671 INFO:scrape:suites union: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2021-03-12T08:35:57.672 INFO:scrape: 2021-03-12T08:35:57.672 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2021-03-12T08:35:57.673 INFO:scrape:2 jobs: ['5956640', '5956799'] 2021-03-12T08:35:57.673 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2021-03-12T08:35:57.674 INFO:scrape:suites union: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2021-03-12T08:35:57.674 INFO:scrape: 2021-03-12T08:35:57.675 INFO:scrape:Timeout 120 running ceph --cluster ceph --log-early log 2021-03-12T08:35:57.675 INFO:scrape:1 jobs: ['5956578'] 2021-03-12T08:35:57.676 INFO:scrape:suites: ['msgr-failures/few', 'msgr/async', 'no_pools', 'objectstore/bluestore-bitmap', 'rados', 'rados/multimon/{clusters/3', 'supported-random-distro$/{centos_latest}', 'tasks/mon_recovery}'] 2021-03-12T08:35:57.676 INFO:scrape: 2021-03-12T08:35:57.676 INFO:scrape:Dead: b'2021-03-12T08:30:54.961 DEBUG:teuthology.exit:Finished running handlers' 2021-03-12T08:35:57.677 INFO:scrape:1 jobs: ['5956659'] 2021-03-12T08:35:57.677 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/fast', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_latest}', 'thrashers/minsize_recovery', 'thrashosds-health', 'workloads/ec-small-objects-balanced}'] 2021-03-12T08:35:57.678 INFO:scrape: 2021-03-12T08:35:57.678 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2021-03-12T08:35:57.679 INFO:scrape:1 jobs: ['5956751'] 2021-03-12T08:35:57.679 INFO:scrape:suites: ['msgr-failures/few', 'msgr/async', 'objectstore/filestore-xfs', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{rhel_latest}}'] 2021-03-12T08:35:57.680 INFO:scrape: 2021-03-12T08:35:57.680 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2021-03-12T08:35:57.680 INFO:scrape:1 jobs: ['5956632'] 2021-03-12T08:35:57.681 INFO:scrape:suites: ['2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_latest}', 'fixed-2}', 'rados/cephadm/upgrade/{1-start'] 2021-03-12T08:35:57.681 INFO:scrape: 2021-03-12T08:35:57.694 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/ failed: 18 dead: 5 running: 0 waiting: 0 queued: 0 passed: 297 Fail ================================================================= [5956516] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} ----------------------------------------------------------------- time: 00:05:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956516/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956516/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ae7d9c74c814e238ddd68150a964e36 Command failed on smithi086 with status 5: 'sudo systemctl stop ceph-e45b9d82-829e-11eb-9085-001a4aab830c@mon.a' [5956523] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:04:32 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956523/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956523/ sentry: https://sentry.ceph.com/organizations/ceph/?query=837e544e37a4429792cc96d643f32ba7 Command failed on smithi046 with status 5: 'sudo systemctl stop ceph-8c343eb4-82a0-11eb-9085-001a4aab830c@mon.smithi046' [5956546] rados/upgrade/mimic-x-singleton/{0-cluster/{openstack start} 1-install/mimic 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} ----------------------------------------------------------------- time: 05:05:38 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956546/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956546/ "2021-03-11T21:30:03.440583+0000 mon.a (mon.0) 12 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED)" in cluster log [5956564] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{rhel_latest} tasks/progress} ----------------------------------------------------------------- time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956564/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956564/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bdccf0e1b2734906b3a0e31fb5d7c7a8 Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress) [5956578] rados/multimon/{clusters/3 msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/mon_recovery} ----------------------------------------------------------------- time: 00:10:52 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956578/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956578/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2e973a6012634e56ad81f73d73c8786d Command failed on smithi017 with status 124: "sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log '1 of 30'" [5956591] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} ----------------------------------------------------------------- time: 00:06:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956591/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956591/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eacf715a8bb64131a833f63f2dcfcf43 Command failed (workunit test cephadm/test_cephadm.sh) on smithi146 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5956618] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:07:00 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956618/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956618/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dcdc4e3fc6234ee0827eef50957315f1 Command failed on smithi017 with status 5: 'sudo systemctl stop ceph- bc100ec2-82a7-11eb-9085-001a4aab830c@mon.a' [5956622] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} ----------------------------------------------------------------- time: 00:05:27 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956622/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956622/ sentry: https://sentry.ceph.com/organizations/ceph/?query=24774825fe6f4638bbd9cbb2c6e189fd Command failed on smithi077 with status 5: 'sudo systemctl stop ceph-90c3c15a-82a7-11eb-9085-001a4aab830c@mon.a' [5956632] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2} ----------------------------------------------------------------- time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956632/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956632/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6c189ae41eb411d89562550aea9f995 reached maximum tries (180) after waiting for 180 seconds [5956640] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} ----------------------------------------------------------------- time: 00:05:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956640/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956640/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9254073d7ee344ec97bf2021ddd342b8 Command failed (workunit test cephadm/test_adoption.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5956677] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} ----------------------------------------------------------------- time: 00:05:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956677/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956677/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b1a268ff7234568b65cdff5606935a1 Command failed on smithi039 with status 5: 'sudo systemctl stop ceph-854797fe-82aa-11eb-9085-001a4aab830c@mon.a' [5956684] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:04:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956684/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956684/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15961152f2e94e4bb11be51eecd6444f Command failed on smithi097 with status 5: 'sudo systemctl stop ceph-9ece5370-82aa-11eb-9085-001a4aab830c@mon.smithi097' [5956746] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} ----------------------------------------------------------------- time: 00:06:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956746/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956746/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bed77cb66e114b6e8dcea91815181471 Command failed (workunit test cephadm/test_cephadm.sh) on smithi075 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5956750] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:07:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956750/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956750/ sentry: https://sentry.ceph.com/organizations/ceph/?query=49efb3d4c4d943fb9ead47bc81d89221 Command failed on smithi005 with status 5: 'sudo systemctl stop ceph-69409cae-82af-11eb-9085-001a4aab830c@mon.a' [5956751] rados/singleton/{all/thrash_cache_writeback_proxy_none msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_latest}} ----------------------------------------------------------------- time: 00:13:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956751/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956751/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5956778] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_latest} tasks/progress} ----------------------------------------------------------------- time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956778/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956778/ sentry: https://sentry.ceph.com/organizations/ceph/?query=57d2e79d80194432b869765d02d07b57 Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress) [5956781] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} ----------------------------------------------------------------- time: 00:05:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956781/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956781/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e11f14772e84daf901d11da57a0ad05 Command failed on smithi137 with status 5: 'sudo systemctl stop ceph-8c6830cc-82b2-11eb-9085-001a4aab830c@mon.a' [5956799] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} ----------------------------------------------------------------- time: 00:06:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956799/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956799/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9f243511c7554a70b0b9a0cb01e13092 Command failed (workunit test cephadm/test_adoption.sh) on smithi141 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e61b15880d42cb4248c17ea9494bc11be2b7a78 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' Dead ================================================================= [5956605] rados/cephadm/smoke/{distro/rhel_7 fixed-2 start} ----------------------------------------------------------------- time: 00:03:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956605/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956605/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6ec53234de6646c68801967fa7739631 {'Failure object was': {'smithi098.front.sepia.ceph.com': {'changed': False, 'results': ['yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed', 'yum-plugin- fastestmirror-1.1.31-52.el7.noarch providing yum-plugin-fastestmirror is already installed', 'redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed', 'sysstat-10.1.5-18.el7.x86_64 providing sysstat is already installed', 'gdb-7.6.1-115.el7.x86_64 providing gdb is already installed', 'git-all-1.8.3.1-20.el7.noarch providing git-all is already installed', 'python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed', 'gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed', 'libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed', 'openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed', 'boost- thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed', 'xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed', 'xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed', 'gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed', 'parted-3.1-31.el7.x86_64 providing parted is already installed', 'libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed', 'fuse-2.9.2-11.el7.x86_64 providing fuse is already installed', 'fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed', '7:lvm2-2.02.185-2.el7.x86_64 providing lvm2 is already installed', '1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed', 'libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed', 'btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed', 'attr-2.4.46-13.el7.x86_64 providing attr is already installed', '1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed', 'python-nose-1.3.7-1.el7.noarch providing python-nose is already installed', 'mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed', 'ant-1.9.4-2.el7.noarch providing ant is already installed', 'lsof-4.87-6.el7.x86_64 providing lsof is already installed', 'iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed', 'libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed', 'automake-1.13.4-3.el7.noarch providing automake is already installed', 'gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed', 'libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed', 'libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed', 'bc-1.06.95-13.el7.x86_64 providing bc is already installed', 'xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed', 'blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed', '1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed', 'python-matplotlib-1.2.0-16.el7.x86_64 providing python- matplotlib is already installed', "No package matching 'qemu-kvm' found available, installed or updated"], 'rc': 126, 'invocation': {'module_args': {'lock_timeout': 30, 'update_cache': False, 'disable_excludes': 'None', 'exclude': [], 'allow_downgrade': False, 'disable_gpg_check': False, 'conf_file': 'None', 'use_backend': 'auto', 'state': 'present', 'disablerepo': [], 'releasever': 'None', 'skip_broken': False, 'autoremove': False, 'download_dir': 'None', 'enable_plugin': [], 'installroot': '/', 'install_weak_deps': True, 'name': ['@core', '@base', 'yum-plugin-priorities', 'yum-plugin-fastestmirror', 'redhat-lsb', 'sysstat', 'gdb', 'git-all', 'python-configobj', 'gcc-c++', 'libedit', 'openssl098e', 'boost-thread', 'xfsprogs', 'xfsprogs-devel', 'gdisk', 'parted', 'libgcrypt', 'fuse', 'fuse-libs', 'lvm2', 'openssl', 'libuuid', 'btrfs-progs', 'attr', 'valgrind', 'python-nose', 'mpich', 'ant', 'lsof', 'iozone', 'libtool', 'automake', 'gettext', 'libuuid-devel', 'libacl- devel', 'bc', 'xfsdump', 'blktrace', 'numpy', 'python-matplotlib', 'qemu- kvm', 'usbredir', 'genisoimage', 'httpd', 'httpd-devel', 'httpd-tools', 'mod_ssl', 'mod_fastcgi-2.4.7-1.ceph.el7', 'libev-devel', 'perl-XML-Twig', 'java-1.6.0-openjdk-devel', 'junit4', 'nfs-utils', 'ncurses-devel', 'python-devel', 'python-virtualenv', 'perl-CPAN', 'podman', 'podman- docker', 'python3'], 'download_only': False, 'bugfix': False, 'list': 'None', 'install_repoquery': True, 'update_only': False, 'disable_plugin': [], 'enablerepo': [], 'security': False, 'validate_certs': True}}, 'msg': "No package matching 'qemu-kvm' found available, installed or updated", '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'changed')"} [5956609] rados/cephadm/smoke-roleless/{distro/rhel_7 start} ----------------------------------------------------------------- time: 00:03:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956609/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956609/ sentry: https://sentry.ceph.com/organizations/ceph/?query=566117ee2bb74833bf904241cdb0d8a8 {'Failure object was': {'smithi075.front.sepia.ceph.com': {'changed': False, 'results': ['yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed', 'yum-plugin- fastestmirror-1.1.31-52.el7.noarch providing yum-plugin-fastestmirror is already installed', 'redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed', 'sysstat-10.1.5-18.el7.x86_64 providing sysstat is already installed', 'gdb-7.6.1-115.el7.x86_64 providing gdb is already installed', 'git-all-1.8.3.1-20.el7.noarch providing git-all is already installed', 'python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed', 'gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed', 'libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed', 'openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed', 'boost- thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed', 'xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed', 'xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed', 'gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed', 'parted-3.1-31.el7.x86_64 providing parted is already installed', 'libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed', 'fuse-2.9.2-11.el7.x86_64 providing fuse is already installed', 'fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed', '7:lvm2-2.02.185-2.el7.x86_64 providing lvm2 is already installed', '1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed', 'libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed', 'btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed', 'attr-2.4.46-13.el7.x86_64 providing attr is already installed', '1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed', 'python-nose-1.3.7-1.el7.noarch providing python-nose is already installed', 'mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed', 'ant-1.9.4-2.el7.noarch providing ant is already installed', 'lsof-4.87-6.el7.x86_64 providing lsof is already installed', 'iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed', 'libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed', 'automake-1.13.4-3.el7.noarch providing automake is already installed', 'gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed', 'libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed', 'libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed', 'bc-1.06.95-13.el7.x86_64 providing bc is already installed', 'xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed', 'blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed', '1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed', 'python-matplotlib-1.2.0-16.el7.x86_64 providing python- matplotlib is already installed', "No package matching 'qemu-kvm' found available, installed or updated"], 'rc': 126, 'invocation': {'module_args': {'lock_timeout': 30, 'update_cache': False, 'disable_excludes': 'None', 'exclude': [], 'allow_downgrade': False, 'disable_gpg_check': False, 'conf_file': 'None', 'use_backend': 'auto', 'state': 'present', 'disablerepo': [], 'releasever': 'None', 'skip_broken': False, 'autoremove': False, 'download_dir': 'None', 'enable_plugin': [], 'installroot': '/', 'install_weak_deps': True, 'name': ['@core', '@base', 'yum-plugin-priorities', 'yum-plugin-fastestmirror', 'redhat-lsb', 'sysstat', 'gdb', 'git-all', 'python-configobj', 'gcc-c++', 'libedit', 'openssl098e', 'boost-thread', 'xfsprogs', 'xfsprogs-devel', 'gdisk', 'parted', 'libgcrypt', 'fuse', 'fuse-libs', 'lvm2', 'openssl', 'libuuid', 'btrfs-progs', 'attr', 'valgrind', 'python-nose', 'mpich', 'ant', 'lsof', 'iozone', 'libtool', 'automake', 'gettext', 'libuuid-devel', 'libacl- devel', 'bc', 'xfsdump', 'blktrace', 'numpy', 'python-matplotlib', 'qemu- kvm', 'usbredir', 'genisoimage', 'httpd', 'httpd-devel', 'httpd-tools', 'mod_ssl', 'mod_fastcgi-2.4.7-1.ceph.el7', 'libev-devel', 'perl-XML-Twig', 'java-1.6.0-openjdk-devel', 'junit4', 'nfs-utils', 'ncurses-devel', 'python-devel', 'python-virtualenv', 'perl-CPAN', 'podman', 'podman- docker', 'python3'], 'download_only': False, 'bugfix': False, 'list': 'None', 'install_repoquery': True, 'update_only': False, 'disable_plugin': [], 'enablerepo': [], 'security': False, 'validate_certs': True}}, 'msg': "No package matching 'qemu-kvm' found available, installed or updated", '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'changed')"} [5956659] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956659/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956659/ hit max job timeout [5956765] rados/cephadm/smoke/{distro/rhel_7 fixed-2 start} ----------------------------------------------------------------- time: 00:03:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956765/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956765/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5585c2da4dbc45639e076b83467dd915 {'Failure object was': {'smithi045.front.sepia.ceph.com': {'changed': False, 'results': ['yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed', 'yum-plugin- fastestmirror-1.1.31-52.el7.noarch providing yum-plugin-fastestmirror is already installed', 'redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed', 'sysstat-10.1.5-18.el7.x86_64 providing sysstat is already installed', 'gdb-7.6.1-115.el7.x86_64 providing gdb is already installed', 'git-all-1.8.3.1-20.el7.noarch providing git-all is already installed', 'python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed', 'gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed', 'libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed', 'openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed', 'boost- thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed', 'xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed', 'xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed', 'gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed', 'parted-3.1-31.el7.x86_64 providing parted is already installed', 'libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed', 'fuse-2.9.2-11.el7.x86_64 providing fuse is already installed', 'fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed', '7:lvm2-2.02.185-2.el7.x86_64 providing lvm2 is already installed', '1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed', 'libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed', 'btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed', 'attr-2.4.46-13.el7.x86_64 providing attr is already installed', '1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed', 'python-nose-1.3.7-1.el7.noarch providing python-nose is already installed', 'mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed', 'ant-1.9.4-2.el7.noarch providing ant is already installed', 'lsof-4.87-6.el7.x86_64 providing lsof is already installed', 'iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed', 'libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed', 'automake-1.13.4-3.el7.noarch providing automake is already installed', 'gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed', 'libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed', 'libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed', 'bc-1.06.95-13.el7.x86_64 providing bc is already installed', 'xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed', 'blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed', '1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed', 'python-matplotlib-1.2.0-16.el7.x86_64 providing python- matplotlib is already installed', "No package matching 'qemu-kvm' found available, installed or updated"], 'rc': 126, 'invocation': {'module_args': {'lock_timeout': 30, 'update_cache': False, 'disable_excludes': 'None', 'exclude': [], 'allow_downgrade': False, 'disable_gpg_check': False, 'conf_file': 'None', 'use_backend': 'auto', 'state': 'present', 'disablerepo': [], 'releasever': 'None', 'skip_broken': False, 'autoremove': False, 'download_dir': 'None', 'enable_plugin': [], 'installroot': '/', 'install_weak_deps': True, 'name': ['@core', '@base', 'yum-plugin-priorities', 'yum-plugin-fastestmirror', 'redhat-lsb', 'sysstat', 'gdb', 'git-all', 'python-configobj', 'gcc-c++', 'libedit', 'openssl098e', 'boost-thread', 'xfsprogs', 'xfsprogs-devel', 'gdisk', 'parted', 'libgcrypt', 'fuse', 'fuse-libs', 'lvm2', 'openssl', 'libuuid', 'btrfs-progs', 'attr', 'valgrind', 'python-nose', 'mpich', 'ant', 'lsof', 'iozone', 'libtool', 'automake', 'gettext', 'libuuid-devel', 'libacl- devel', 'bc', 'xfsdump', 'blktrace', 'numpy', 'python-matplotlib', 'qemu- kvm', 'usbredir', 'genisoimage', 'httpd', 'httpd-devel', 'httpd-tools', 'mod_ssl', 'mod_fastcgi-2.4.7-1.ceph.el7', 'libev-devel', 'perl-XML-Twig', 'java-1.6.0-openjdk-devel', 'junit4', 'nfs-utils', 'ncurses-devel', 'python-devel', 'python-virtualenv', 'perl-CPAN', 'podman', 'podman- docker', 'python3'], 'download_only': False, 'bugfix': False, 'list': 'None', 'install_repoquery': True, 'update_only': False, 'disable_plugin': [], 'enablerepo': [], 'security': False, 'validate_certs': True}}, 'msg': "No package matching 'qemu-kvm' found available, installed or updated", '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'changed')"} [5956769] rados/cephadm/smoke-roleless/{distro/rhel_7 start} ----------------------------------------------------------------- time: 00:03:41 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956769/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956769/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a20fd6d421e747e1aa5e9cf7848501c4 {'Failure object was': {'smithi005.front.sepia.ceph.com': {'changed': False, 'results': ['yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed', 'yum-plugin- fastestmirror-1.1.31-52.el7.noarch providing yum-plugin-fastestmirror is already installed', 'redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed', 'sysstat-10.1.5-18.el7.x86_64 providing sysstat is already installed', 'gdb-7.6.1-115.el7.x86_64 providing gdb is already installed', 'git-all-1.8.3.1-20.el7.noarch providing git-all is already installed', 'python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed', 'gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed', 'libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed', 'openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed', 'boost- thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed', 'xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed', 'xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed', 'gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed', 'parted-3.1-31.el7.x86_64 providing parted is already installed', 'libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed', 'fuse-2.9.2-11.el7.x86_64 providing fuse is already installed', 'fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed', '7:lvm2-2.02.185-2.el7.x86_64 providing lvm2 is already installed', '1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed', 'libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed', 'btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed', 'attr-2.4.46-13.el7.x86_64 providing attr is already installed', '1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed', 'python-nose-1.3.7-1.el7.noarch providing python-nose is already installed', 'mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed', 'ant-1.9.4-2.el7.noarch providing ant is already installed', 'lsof-4.87-6.el7.x86_64 providing lsof is already installed', 'iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed', 'libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed', 'automake-1.13.4-3.el7.noarch providing automake is already installed', 'gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed', 'libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed', 'libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed', 'bc-1.06.95-13.el7.x86_64 providing bc is already installed', 'xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed', 'blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed', '1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed', 'python-matplotlib-1.2.0-16.el7.x86_64 providing python- matplotlib is already installed', "No package matching 'qemu-kvm' found available, installed or updated"], 'rc': 126, 'invocation': {'module_args': {'lock_timeout': 30, 'update_cache': False, 'disable_excludes': 'None', 'exclude': [], 'allow_downgrade': False, 'disable_gpg_check': False, 'conf_file': 'None', 'use_backend': 'auto', 'state': 'present', 'disablerepo': [], 'releasever': 'None', 'skip_broken': False, 'autoremove': False, 'download_dir': 'None', 'enable_plugin': [], 'installroot': '/', 'install_weak_deps': True, 'name': ['@core', '@base', 'yum-plugin-priorities', 'yum-plugin-fastestmirror', 'redhat-lsb', 'sysstat', 'gdb', 'git-all', 'python-configobj', 'gcc-c++', 'libedit', 'openssl098e', 'boost-thread', 'xfsprogs', 'xfsprogs-devel', 'gdisk', 'parted', 'libgcrypt', 'fuse', 'fuse-libs', 'lvm2', 'openssl', 'libuuid', 'btrfs-progs', 'attr', 'valgrind', 'python-nose', 'mpich', 'ant', 'lsof', 'iozone', 'libtool', 'automake', 'gettext', 'libuuid-devel', 'libacl- devel', 'bc', 'xfsdump', 'blktrace', 'numpy', 'python-matplotlib', 'qemu- kvm', 'usbredir', 'genisoimage', 'httpd', 'httpd-devel', 'httpd-tools', 'mod_ssl', 'mod_fastcgi-2.4.7-1.ceph.el7', 'libev-devel', 'perl-XML-Twig', 'java-1.6.0-openjdk-devel', 'junit4', 'nfs-utils', 'ncurses-devel', 'python-devel', 'python-virtualenv', 'perl-CPAN', 'podman', 'podman- docker', 'python3'], 'download_only': False, 'bugfix': False, 'list': 'None', 'install_repoquery': True, 'update_only': False, 'disable_plugin': [], 'enablerepo': [], 'security': False, 'validate_certs': True}}, 'msg': "No package matching 'qemu-kvm' found available, installed or updated", '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500 a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'changed')"} Pass ================================================================= [5956495] rados/singleton/{all/dump-stuck msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest}} time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956495/ [5956496] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956496/ [5956497] rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956497/ [5956498] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:22:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956498/ [5956499] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956499/ [5956500] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:11:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956500/ [5956501] rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} time: 01:27:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956501/ [5956502] rados/singleton/{all/ec-lost-unfound msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest}} time: 00:34:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956502/ [5956503] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/rados_python} time: 00:13:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956503/ [5956504] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956504/ [5956505] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup_tier} time: 00:09:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956505/ [5956506] rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956506/ [5956507] rados/singleton-nomsgr/{all/recovery-unfound-found rados supported-random-distro$/{rhel_latest}} time: 00:29:35 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956507/ [5956508] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_api_tests} time: 00:30:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956508/ [5956509] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{rhel_latest} tasks/dashboard} time: 01:07:22 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956509/ [5956510] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 04:24:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956510/ [5956511] rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956511/ [5956512] rados/singleton/{all/erasure-code-nonregression msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest}} time: 00:06:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956512/ [5956513] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} time: 00:04:30 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956513/ [5956514] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:30:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956514/ [5956515] rados/monthrash/{ceph clusters/9-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/many workloads/pool-create-delete} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956515/ [5956517] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:19:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956517/ [5956518] rados/multimon/{clusters/9 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:06:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956518/ [5956519] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956519/ [5956520] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956520/ [5956521] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:25:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956521/ [5956522] rados/singleton/{all/lost-unfound-delete msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956522/ [5956524] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956524/ [5956525] rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956525/ [5956526] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async start tasks/rados_python} time: 00:19:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956526/ [5956527] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:24:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956527/ [5956528] rados/singleton-nomsgr/{all/version-number-sanity rados supported-random-distro$/{rhel_latest}} time: 00:10:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956528/ [5956529] rados/singleton/{all/lost-unfound msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest}} time: 00:27:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956529/ [5956530] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956530/ [5956531] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956531/ [5956532] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} tasks/module_selftest} time: 00:30:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956532/ [5956533] rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} time: 00:16:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956533/ [5956534] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956534/ [5956535] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:56 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956535/ [5956536] rados/singleton/{all/max-pg-per-osd.from-mon msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956536/ [5956537] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:33:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956537/ [5956538] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956538/ [5956539] rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:22 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956539/ [5956540] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_api_tests} time: 00:37:40 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956540/ [5956541] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/dashboard} time: 01:06:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956541/ [5956542] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:06:43 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956542/ [5956543] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:07:31 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956543/ [5956544] rados/singleton-nomsgr/{all/admin_socket_output rados supported-random-distro$/{centos_latest}} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956544/ [5956545] rados/standalone/{supported-random-distro$/{centos_latest} workloads/crush} time: 00:09:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956545/ [5956547] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956547/ [5956548] rados/singleton/{all/max-pg-per-osd.from-primary msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:12:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956548/ [5956549] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:36:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956549/ [5956550] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956550/ [5956551] rados/monthrash/{ceph clusters/3-mons msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_5925} time: 00:07:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956551/ [5956552] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956552/ [5956553] rados/multimon/{clusters/21 msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:06:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956553/ [5956554] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956554/ [5956555] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:42 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956555/ [5956556] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:21:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956556/ [5956557] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956557/ [5956558] rados/cephadm/smoke/{distro/centos_7 fixed-2 start} time: 00:16:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956558/ [5956559] rados/singleton/{all/max-pg-per-osd.from-replica msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:11:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956559/ [5956560] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} tasks/rados_striper} time: 00:10:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956560/ [5956561] rados/cephadm/smoke-roleless/{distro/centos_7 start} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956561/ [5956562] rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956562/ [5956563] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956563/ [5956565] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_7} fixed-2} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956565/ [5956566] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:43 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956566/ [5956567] rados/singleton/{all/mon-auth-caps msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest}} time: 00:10:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956567/ [5956568] rados/singleton-nomsgr/{all/balancer rados supported-random-distro$/{rhel_latest}} time: 00:11:23 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956568/ [5956569] rados/cephadm/workunits/{distro/centos_latest task/test_adoption} time: 00:07:42 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956569/ [5956570] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:10:02 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956570/ [5956571] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_python} time: 00:23:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956571/ [5956572] rados/singleton/{all/mon-config-key-caps msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:08:13 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956572/ [5956573] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956573/ [5956574] rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956574/ [5956575] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:40 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956575/ [5956576] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:57 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956576/ [5956577] rados/monthrash/{ceph clusters/9-mons msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/sync-many workloads/rados_api_tests} time: 00:26:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956577/ [5956579] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_latest}} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956579/ [5956580] rados/singleton/{all/mon-config-keys msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest}} time: 00:15:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956580/ [5956581] rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:28:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956581/ [5956582] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:10:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956582/ [5956583] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:14 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956583/ [5956584] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956584/ [5956585] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956585/ [5956586] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956586/ [5956587] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/jewel-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 00:54:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956587/ [5956588] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:38:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956588/ [5956589] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} tasks/rados_workunit_loadgen_big} time: 00:32:17 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956589/ [5956590] rados/singleton-nomsgr/{all/cache-fs-trunc rados supported-random-distro$/{centos_latest}} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956590/ [5956592] rados/cephadm/smoke/{distro/centos_latest fixed-2 start} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956592/ [5956593] rados/singleton/{all/mon-config msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:07:19 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956593/ [5956594] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} tasks/prometheus} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956594/ [5956595] rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} time: 00:34:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956595/ [5956596] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/set-chunk-promote-flush} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956596/ [5956597] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:11:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956597/ [5956598] rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:40 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956598/ [5956599] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:28:00 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956599/ [5956600] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async start tasks/rados_api_tests} time: 00:30:20 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956600/ [5956601] rados/singleton/{all/osd-backfill msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:19:31 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956601/ [5956602] rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} time: 00:04:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956602/ [5956603] rados/singleton-nomsgr/{all/ceph-kvstore-tool rados supported-random-distro$/{ubuntu_latest}} time: 00:07:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956603/ [5956604] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:15:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956604/ [5956606] rados/singleton/{all/osd-recovery-incomplete msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest}} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956606/ [5956607] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956607/ [5956608] rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:16:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956608/ [5956610] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956610/ [5956611] rados/multimon/{clusters/6 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:06:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956611/ [5956612] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956612/ [5956613] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:17 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956613/ [5956614] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:12:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956614/ [5956615] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/jewel backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:20:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956615/ [5956616] rados/monthrash/{ceph clusters/3-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:38:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956616/ [5956617] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:20:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956617/ [5956619] rados/singleton/{all/osd-recovery msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest}} time: 00:14:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956619/ [5956620] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} tasks/dashboard} time: 01:05:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956620/ [5956621] rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} time: 00:05:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956621/ [5956623] rados/singleton-nomsgr/{all/ceph-post-file rados supported-random-distro$/{ubuntu_latest}} time: 00:05:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956623/ [5956624] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:06:48 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956624/ [5956625] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956625/ [5956626] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:20:23 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956626/ [5956627] rados/singleton/{all/peer msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:06:38 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956627/ [5956628] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956628/ [5956629] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 01:45:27 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956629/ [5956630] rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956630/ [5956631] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956631/ [5956633] rados/singleton/{all/pg-autoscaler-progress-off msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:08:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956633/ [5956634] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956634/ [5956635] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects} time: 00:17:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956635/ [5956636] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} time: 00:21:22 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956636/ [5956637] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:09:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956637/ [5956638] rados/singleton-nomsgr/{all/export-after-evict rados supported-random-distro$/{centos_latest}} time: 00:07:18 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956638/ [5956639] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956639/ [5956641] rados/singleton/{all/pg-autoscaler msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:08:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956641/ [5956642] rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956642/ [5956643] rados/multimon/{clusters/9 msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956643/ [5956644] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956644/ [5956645] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956645/ [5956646] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:24:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956646/ [5956647] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:36:27 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956647/ [5956648] rados/monthrash/{ceph clusters/9-mons msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:25:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956648/ [5956649] rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956649/ [5956650] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:31:45 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956650/ [5956651] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956651/ [5956652] rados/singleton/{all/pg-removal-interruption msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:08:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956652/ [5956653] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} tasks/dashboard} time: 01:05:42 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956653/ [5956654] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:09:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956654/ [5956655] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{ubuntu_latest}} time: 02:18:48 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956655/ [5956656] rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest}} time: 00:32:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956656/ [5956657] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:18:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956657/ [5956658] rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} time: 00:12:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956658/ [5956660] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956660/ [5956661] rados/singleton-nomsgr/{all/full-tiering rados supported-random-distro$/{rhel_latest}} time: 00:10:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956661/ [5956662] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_api_tests} time: 00:28:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956662/ [5956663] rados/singleton/{all/radostool msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956663/ [5956664] rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956664/ [5956665] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:10:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956665/ [5956666] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956666/ [5956667] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956667/ [5956668] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956668/ [5956669] rados/singleton/{all/random-eio msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_latest}} time: 00:19:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956669/ [5956670] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:02 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956670/ [5956671] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956671/ [5956672] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} time: 00:04:38 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956672/ [5956673] rados/multimon/{clusters/21 msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} tasks/mon_recovery} time: 00:25:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956673/ [5956674] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956674/ [5956675] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956675/ [5956676] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:20:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956676/ [5956678] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:19:38 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956678/ [5956679] rados/monthrash/{ceph clusters/3-mons msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:27:23 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956679/ [5956680] rados/singleton-nomsgr/{all/health-warnings rados supported-random-distro$/{ubuntu_latest}} time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956680/ [5956681] rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956681/ [5956682] rados/singleton/{all/rebuild-mondb msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest}} time: 00:15:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956682/ [5956683] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956683/ [5956685] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/failover} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956685/ [5956686] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:22:08 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956686/ [5956687] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} time: 00:24:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956687/ [5956688] rados/singleton/{all/recovery-preemption msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956688/ [5956689] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:17 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956689/ [5956690] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_latest}} time: 02:16:32 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956690/ [5956691] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} tasks/repair_test} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956691/ [5956692] rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} time: 00:16:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956692/ [5956693] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956693/ [5956694] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:26:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956694/ [5956695] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} time: 00:13:17 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956695/ [5956696] rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956696/ [5956697] rados/singleton/{all/resolve_stuck_peering msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956697/ [5956698] rados/singleton-nomsgr/{all/large-omap-object-warnings rados supported-random-distro$/{centos_latest}} time: 00:07:31 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956698/ [5956699] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:53 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956699/ [5956700] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956700/ [5956701] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956701/ [5956702] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} time: 00:30:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956702/ [5956703] rados/singleton/{all/test-crash msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:09:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956703/ [5956704] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} time: 03:28:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956704/ [5956705] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_latest}} time: 00:13:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956705/ [5956706] rados/multimon/{clusters/3 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} tasks/mon_clock_no_skews} time: 00:09:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956706/ [5956707] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956707/ [5956708] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956708/ [5956709] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 01:02:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956709/ [5956710] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:34:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956710/ [5956711] rados/monthrash/{ceph clusters/9-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:35:43 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956711/ [5956712] rados/cephadm/smoke/{distro/centos_7 fixed-2 start} time: 00:16:21 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956712/ [5956713] rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:13:18 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956713/ [5956714] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:11:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956714/ [5956715] rados/singleton/{all/test_envlibrados_for_rocksdb msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:13:14 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956715/ [5956716] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:12:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956716/ [5956717] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/scrub_test} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956717/ [5956718] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956718/ [5956719] rados/singleton-nomsgr/{all/lazy_omap_stats_output rados supported-random-distro$/{ubuntu_latest}} time: 00:08:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956719/ [5956720] rados/cephadm/smoke-roleless/{distro/centos_7 start} time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956720/ [5956721] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_7} fixed-2} time: 00:19:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956721/ [5956722] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:21:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956722/ [5956723] rados/singleton/{all/thrash-backfill-full msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:51:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956723/ [5956724] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956724/ [5956725] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_latest}} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956725/ [5956726] rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest}} time: 00:31:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956726/ [5956727] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956727/ [5956728] rados/cephadm/workunits/{distro/centos_latest task/test_adoption} time: 00:07:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956728/ [5956729] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:42:16 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956729/ [5956730] rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956730/ [5956731] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956731/ [5956732] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} time: 00:21:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956732/ [5956733] rados/singleton/{all/thrash-eio msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest}} time: 00:41:23 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956733/ [5956734] rados/singleton-nomsgr/{all/librados_hello_world rados supported-random-distro$/{centos_latest}} time: 00:08:38 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956734/ [5956735] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} time: 00:14:05 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956735/ [5956736] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:20:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956736/ [5956737] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956737/ [5956738] rados/multimon/{clusters/6 msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} tasks/mon_clock_with_skews} time: 00:09:23 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956738/ [5956739] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:58 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956739/ [5956740] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956740/ [5956741] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:11:09 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956741/ [5956742] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:18:48 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956742/ [5956743] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:23:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956743/ [5956744] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/rados_api_tests} time: 00:22:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956744/ [5956745] rados/monthrash/{ceph clusters/3-mons msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:16:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956745/ [5956747] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_latest} tasks/module_selftest} time: 00:31:47 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956747/ [5956748] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:19:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956748/ [5956749] rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956749/ [5956752] rados/cephadm/smoke/{distro/centos_latest fixed-2 start} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956752/ [5956753] rados/singleton-nomsgr/{all/msgr rados supported-random-distro$/{ubuntu_latest}} time: 00:15:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956753/ [5956754] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:18:55 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956754/ [5956755] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:11:53 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956755/ [5956756] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:22:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956756/ [5956757] rados/singleton/{all/watch-notify-same-primary msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:07:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956757/ [5956758] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_latest} tasks/dashboard} time: 01:06:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956758/ [5956759] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_latest}} time: 00:07:12 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956759/ [5956760] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:52 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956760/ [5956761] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956761/ [5956762] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:17:24 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956762/ [5956763] rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} time: 00:05:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956763/ [5956764] rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:17 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956764/ [5956766] rados/singleton/{all/admin-socket msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:06:19 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956766/ [5956767] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956767/ [5956768] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache} time: 00:09:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956768/ [5956770] rados/multimon/{clusters/9 msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956770/ [5956771] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956771/ [5956772] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956772/ [5956773] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:37:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956773/ [5956774] rados/singleton-nomsgr/{all/multi-backfill-reject rados supported-random-distro$/{ubuntu_latest}} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956774/ [5956775] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:16:53 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956775/ [5956776] rados/monthrash/{ceph clusters/9-mons msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_5925} time: 00:09:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956776/ [5956777] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956777/ [5956779] rados/standalone/{supported-random-distro$/{rhel_latest} workloads/osd} time: 02:54:28 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956779/ [5956780] rados/singleton/{all/deduptool msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:46 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956780/ [5956782] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/dedup_tier} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956782/ [5956783] rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956783/ [5956784] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} time: 00:16:37 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956784/ [5956785] rados/singleton/{all/divergent_priors msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956785/ [5956786] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:24:25 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956786/ [5956787] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:15:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956787/ [5956788] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:48 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956788/ [5956789] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:11 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956789/ [5956790] rados/singleton-nomsgr/{all/osd_stale_reads rados supported-random-distro$/{centos_latest}} time: 00:17:36 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956790/ [5956791] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} time: 00:29:44 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956791/ [5956792] rados/singleton/{all/divergent_priors2 msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:07:19 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956792/ [5956793] rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/rados_python} time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956793/ [5956794] rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:05:33 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956794/ [5956795] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956795/ [5956796] rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:29:14 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956796/ [5956797] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2} time: 00:18:51 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956797/ [5956798] rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956798/ [5956800] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:23:45 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956800/ [5956801] rados/singleton/{all/dump-stuck msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:09:06 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956801/ [5956802] rados/multimon/{clusters/21 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews} time: 00:07:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956802/ [5956803] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:12 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956803/ [5956804] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:04 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956804/ [5956805] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956805/ [5956806] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:50 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956806/ [5956807] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:35:39 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956807/ [5956808] rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956808/ [5956809] rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} tasks/prometheus} time: 00:11:03 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956809/ [5956810] rados/monthrash/{ceph clusters/3-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:24:30 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956810/ [5956811] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956811/ [5956812] rados/singleton-nomsgr/{all/pool-access rados supported-random-distro$/{ubuntu_latest}} time: 00:06:53 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956812/ [5956813] rados/singleton/{all/ec-lost-unfound msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest}} time: 00:32:54 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956813/ [5956814] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:50:59 info: http://pulpito.ceph.com/yuriw-2021-03-11_19:01:40-rados-octopus-distro-basic-smithi/5956814/